Skip to content

Harden fetchAndTestRpcs parsing against DefiLlama format changes and add RPC fallbacks#638

Merged
L03TJ3 merged 18 commits into
masterfrom
copilot/fix-fetch-and-test-rpcs-failure
May 27, 2026
Merged

Harden fetchAndTestRpcs parsing against DefiLlama format changes and add RPC fallbacks#638
L03TJ3 merged 18 commits into
masterfrom
copilot/fix-fetch-and-test-rpcs-failure

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 20, 2026

Fixes #637

fetchAndTestRpcs was failing due to brittle parsing of DefiLlama's extraRpcs.js source file, which could leave tested RPC sets empty. This update replaces the custom JS parser with a simple JSON fetch and guarantees fallback RPCs when any step fails.

What changed

  • src/hooks/rpcParsing.ts — replaced ~150-line balanced-segment JS parser with a fetchRpcsFromChainlist() function that fetches chainid.network/chains.json (a stable JSON endpoint), filters for required chains (1, 122, 42220, 50), and strips non-HTTP(S) URLs. No custom parsing, no eval(), no regex on JS source.
  • src/hooks/useWeb3.tsx — updated fetchAndTestRpcs to call fetchRpcsFromChainlist() directly, removed the CHAINLIST_URL constant and chainMapping object, and added rpcInitializationPromise = null in the catch block so the app can retry on failure without a page reload.
  • src/hooks/rpcParsing.test.ts — updated to test the new API: live fetch against chainid.network asserting all 4 chains return valid HTTP(S) URLs, plus a unit test for FALLBACK_RPCS_BY_CHAIN.

Test evidence

image

Full suite: 44 tests passing, 7 pre-existing failures unrelated to this PR (@sushiswap/sdk pnpm hoisting, @reown/appkit ESM — both present on master).

pnpm lint — clean.

pnpm build — fails with pre-existing @ethersproject/bignumber rollup resolution error (pnpm hoisting incompatibility, not introduced by this branch).

Summary by Sourcery

Improve RPC discovery and network UX while hardening tests and CI around RPC parsing.

New Features:

  • Introduce a shared fallback RPC mapping for mainnet, Fuse, Celo, and XDC and use it as a default throughout the app.
  • Add a direct on-chain native balance read with a loading spinner as a fallback when the primary balance hook is slow or stuck.

Bug Fixes:

  • Replace brittle DefiLlama source parsing with a stable JSON-based RPC fetch that gracefully falls back when remote data is unavailable.
  • Ensure RPC initialization can retry after failures instead of getting stuck, and always yields at least fallback RPCs.
  • Prevent multiple simultaneous network switches and handle network switching errors more robustly in the network modal.

Enhancements:

  • Standardize network RPC defaults by reusing the shared fallback RPC configuration.
  • Improve the network switch UI with disabled options and a visible "switching network" state.
  • Refine wallet option components to support a disabled state consistent with network switching UX.

Build:

  • Add Babel and Jest configuration plus supporting dev dependencies to better support TypeScript and Node-based Jest tests.

CI:

  • Introduce a Jest GitHub Actions workflow to run the RPC parsing tests on pull requests.

Tests:

  • Add rpcParsing tests that exercise the new chainlist-based RPC fetching and fallback behavior.
  • Configure Jest with a jsdom environment, setup file, and extended timeout to support the new tests.

@github-project-automation github-project-automation Bot moved this to Prepare AI Task in GoodBounties May 20, 2026
Copilot AI linked an issue May 20, 2026 that may be closed by this pull request
2 tasks
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented May 20, 2026

Deploying goodprotocolui with  Cloudflare Pages  Cloudflare Pages

Latest commit: 996a90d
Status: ✅  Deploy successful!
Preview URL: https://49642df0.goodprotocolui.pages.dev
Branch Preview URL: https://copilot-fix-fetch-and-test-r.goodprotocolui.pages.dev

View logs

@sirpy
Copy link
Copy Markdown
Contributor

sirpy commented May 20, 2026

@L03TJ3

Copilot AI changed the title [WIP] Fix bug with fetchAndTestRpcs failing Harden fetchAndTestRpcs parsing against DefiLlama format changes and add RPC fallbacks May 20, 2026
Copilot finished work on behalf of sirpy May 20, 2026 09:03
Copilot AI requested a review from sirpy May 20, 2026 09:03
@GoodDollar GoodDollar deleted a comment from wbrasilsousa May 21, 2026
@Gutopro
Copy link
Copy Markdown
Collaborator

Gutopro commented May 21, 2026

Handoff — ready for review @L03TJ3

What changed from the Copilot PR

The original implementation fetched a raw .js source file from DefiLlama and used a custom ~150-line balanced-segment scanner to extract RPC arrays from it. Per the bounty scope ("heavily simplified, not over-engineered"), I replaced this entirely.

rpcParsing.ts now calls fetchRpcsFromChainlist() which fetches chainid.network/chains.json — a proper JSON endpoint — and filters results to the 4 required chains with HTTP(S) URLs only. The file is ~25 lines. No custom parser, no JS source fetching.

In useWeb3.tsx, added the missing rpcInitializationPromise = null reset in the catch block so the app can retry after a failure without a full page reload. Also removed the now-unnecessary CHAINLIST_URL constant and chainMapping object.

What was tested

  • Live fetch test against chainid.network — all 4 chains (1, 122, 42220, 50) return non-empty HTTP(S) URLs ✓
  • FALLBACK_RPCS_BY_CHAIN unit test — all 4 chains covered with valid URLs ✓
  • pnpm lint — clean ✓
  • Full jest suite — 44 passing, 7 pre-existing failures not introduced by this branch ✓

Remaining risks

  • chainid.network/chains.json schema could change — if the rpc field is renamed, parsing silently returns empty arrays and falls back to constants
  • Fallback RPC URLs need periodic maintenance as endpoints may go offline

@L03TJ3 L03TJ3 removed this from GoodBounties May 22, 2026
@sirpy
Copy link
Copy Markdown
Contributor

sirpy commented May 25, 2026

@L03TJ3 @Gutopro
I dont see we have any tests running on github workflow, perhaps we should add it. so at least we see the rpc tests passsing

@L03TJ3
Copy link
Copy Markdown
Collaborator

L03TJ3 commented May 25, 2026

@sirpy was testing and encountered a bug. pushed a fix though it ended up not being related to rpcs.
if you confirm that its okay if we use the new source for chain lists.

Then the only remaining task is adding to run the test on github workflow.
@Gutopro can you pick that up, know how to do it?

@L03TJ3
Copy link
Copy Markdown
Collaborator

L03TJ3 commented May 26, 2026

@Gutopro why are you enforcing pnpm, the package manager for the repository is yarn?
did you look at the error logs?

@Gutopro
Copy link
Copy Markdown
Collaborator

Gutopro commented May 26, 2026

the test all pass now @L03TJ3

@L03TJ3 L03TJ3 marked this pull request as ready for review May 27, 2026 07:03
@L03TJ3 L03TJ3 requested a review from a team May 27, 2026 07:03
@L03TJ3
Copy link
Copy Markdown
Collaborator

L03TJ3 commented May 27, 2026

going to approve but please follow up on my telegram message @Gutopro

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues, and left some high level feedback:

  • The fallback balance logic in Web3StatusInner mixes nativeBalance (likely already formatted) with directNativeBalance (a raw getBalance value in wei), but both are run through parseFloat(...).toFixed(4); consider normalizing units so the direct RPC balance is converted with the correct decimals before display to avoid misleading amounts.
  • In Web3StatusInner, useEthers is cast to any and library is assumed to have getBalance; it would be more robust to type this hook properly (or narrow the type) so that library can be safely used and refactoring is less error-prone.
  • The Jest setup introduces jest 29.x while jest-environment-jsdom is at 30.x; aligning these versions (or dropping the explicit jest-environment-jsdom devDependency and relying on the built-in environment) will reduce the risk of subtle test runner incompatibilities.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The fallback balance logic in `Web3StatusInner` mixes `nativeBalance` (likely already formatted) with `directNativeBalance` (a raw `getBalance` value in wei), but both are run through `parseFloat(...).toFixed(4)`; consider normalizing units so the direct RPC balance is converted with the correct decimals before display to avoid misleading amounts.
- In `Web3StatusInner`, `useEthers` is cast to `any` and `library` is assumed to have `getBalance`; it would be more robust to type this hook properly (or narrow the type) so that `library` can be safely used and refactoring is less error-prone.
- The Jest setup introduces `jest` 29.x while `jest-environment-jsdom` is at 30.x; aligning these versions (or dropping the explicit `jest-environment-jsdom` devDependency and relying on the built-in environment) will reduce the risk of subtle test runner incompatibilities.

## Individual Comments

### Comment 1
<location path="src/components/Web3Status/index.tsx" line_range="74-78" />
<code_context>
+        }
+    }, [address, library, /*used*/ chainId])
+
+    const displayNativeBalance = nativeBalance ?? directNativeBalance
+    const shouldShowBalance = showBalance && !!displayNativeBalance

</code_context>
<issue_to_address>
**issue (bug_risk):** Direct balance fallback is in base units and will be rendered as a huge, mis-scaled value.

`nativeBalance` from `useNativeBalance` appears to be in human-readable units, but `directNativeBalance` is the raw `library.getBalance(address)` result (a wei `BigNumber` in ethers v5). Using it directly (or via `parseFloat`) will display wei as if it were ETH/FUSE, off by 10^18. Normalize `directNativeBalance` first (e.g. `ethers.utils.formatUnits(balance, decimals)`) so its units match `nativeBalance` before computing `displayNativeBalance`.
</issue_to_address>

### Comment 2
<location path="src/functions/rpcParsing.ts" line_range="20-21" />
<code_context>
+    const result: Record<string, string[]> = {}
+    for (const chain of chains) {
+        if (TARGET_CHAIN_IDS.has(chain.chainId)) {
+            result[String(chain.chainId)] = chain.rpc.filter(
+                (url) => url.startsWith('http://') || url.startsWith('https://')
+            )
+        }
</code_context>
<issue_to_address>
**🚨 suggestion (security):** Including plain HTTP RPC URLs may introduce security/privacy risks.

This currently allows both `http://` and `https://` RPC endpoints. For public/production networks, `http://` risks traffic tampering and data leakage. Consider restricting to `https://` for mainnets, and only allow `http://` explicitly for local/test setups (e.g., behind a feature flag or env-based check).
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +74 to 78
const displayNativeBalance = nativeBalance ?? directNativeBalance
const shouldShowBalance = showBalance && !!displayNativeBalance

const sortedRecentTransactions = useMemo(() => {
const txs = Object.values(allTransactions)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Direct balance fallback is in base units and will be rendered as a huge, mis-scaled value.

nativeBalance from useNativeBalance appears to be in human-readable units, but directNativeBalance is the raw library.getBalance(address) result (a wei BigNumber in ethers v5). Using it directly (or via parseFloat) will display wei as if it were ETH/FUSE, off by 10^18. Normalize directNativeBalance first (e.g. ethers.utils.formatUnits(balance, decimals)) so its units match nativeBalance before computing displayNativeBalance.

Comment thread src/functions/rpcParsing.ts
@L03TJ3 L03TJ3 merged commit 47a17ac into master May 27, 2026
7 checks passed
@L03TJ3 L03TJ3 deleted the copilot/fix-fetch-and-test-rpcs-failure branch May 27, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Plan] Bug: fetchAndTestRpcs failing

4 participants